The present invention relates to representing and analyzing sequences of images of a natural scene.
It has been proposed that natural images can be described in terms of intrinsic characteristics, such as range, orientation, reflectance, and incident illumination of the surface element visible at each point in the image H. G. Barrow & J. M. Tenenbaum, Recovering intrinsic scene characteristics from images, in A. Hanson & E. Riseman, editors, Computer Vision Systems, Academic Press (1978). The extraction of information describing such characteristics is complicated, however, by the fact that the information representing the combined characteristics for each pixel in an image is confounded ID a single pixel value representing the intensity of light captured for the corresponding location. The decomposition of these pixel values to obtain information corresponding to intrinsic characteristics depends on the introduction of constraints derived from assumptions about the scene and the imaging process.
In one approach to the problem of decomposing an image sequence including t images into constant reflectance and varying illumination such that:It=Rt  (1)the images I′ are first transformed into the log domain where their compositions as component-wise products of reflectance and illumination are replaced by component-wise sums of corresponding logged terms:it=r+lt  (2)where it, r, and lt denote the logs of lt, R, and Lt.
Vertical and horizontal derivative filters f1 and f2 are then applied to i′:int=rn+lnt  (3)where int, rn, and lnt denote the convolutions fn*it, fn*r, and fn*lt. When applied to natural images, the outputs of these filters tend to be sparse. That is, the probability distributions of their outputs are peaked at zero and fall off much more rapidly than Gaussian. Under the assumption that the illumination term lnt in equation (3) is sparse, an estimate {circumflex over (r)}n of rn is obtained by applying the median over time to int:{circumflex over (r)}n=mediant(int)=mediant(rn+lnt)  (4)This estimate follows from the observation that if lnt at some pixel is within ε of zero more than 50% of the time, then by definition the median over time of rn+lnt at that pixel will be within ε of rn.
Finally, an estimate {circumflex over (r)} of r is reconstructed by solving the over-constrained linear system:fn*{circumflex over (r)}={circumflex over (r)}n  (5)An image {circumflex over (r)} whose filter outputs match those determined by the estimator in equation (4) is sought.
A pseudo-inverse solution is employed to recover the best {circumflex over (r)} in the least squared error sense. The solution is given by:
                              r          ^                =                  g          *                      (                                          ∑                n                            ⁢                                                          ⁢                                                f                  n                  r                                *                                                      r                    ^                                    n                                                      )                                              (        6        )            with fnr the reversed filter of fn and g a solution to:
                              g          *                      (                                          ∑                n                            ⁢                                                          ⁢                                                f                  n                  r                                *                                  f                  n                                                      )                          =        δ                            (        7        )            
Due to the DC-free nature of the filters, a DC term for {circumflex over (r)} is not recovered by equation (6). This term is set equal to the median over time of the DC terms of it. Estimates {circumflex over (l)}t of illumination lt are found by rewriting equation (2) as:{circumflex over (l)}t=it−{circumflex over (r)}  (8)
Estimates {circumflex over (R)} and {circumflex over (L)}t of R and Lt are found by applying exponents to {circumflex over (r)} and {circumflex over (l)}t.